This disclosure includes Appendices A, B and C, which have not been printed with the specification but are available in the application file. Appendices A and B are software for carrying out the invention. Appendix C is a copy of application Ser. No. 07/452,960, filed Dec. 19, 1989 and incorporated herein by reference.
This application relates to digital data processing and, more particularly, relates to methods and apparatus for improved data migration.
The use of storage-intensive computer applications such as high-performance, high-resolution graphics has grown significantly in recent years, with indications that it will continue to grow through the next decade. Fueling user demand has been the introduction of lower cost 32-bit workstations and an increase in the base of applications software available for those systems. Because of their computational and graphics power, these workstations are employed in data-intensive applications such as electronic publishing, computer-aided design (CAD) and scientific research.
Paralleling these developments has been the emergence of industry standard communication protocols which permit users to operate in a multi-vendor environment. Of particular interest is the Network File System (NFS) protocol, developed by Sun Microsystems, allowing users to share files across a local area network such as Ethernet.
Thus, over the past few years, available CPU power per site has grown by several orders of magnitude, due largely to the introduction of engineering workstations and personal computers. The growth in secondary storage capacity, however, has not kept pace. Data intensive applications such as mechanical CAD, electrical CAD, scientific visualization and computer generated animation are straining the data storage capability of existing file servers. The data storage requirements of professional work groups employing networked workstations can measure billions of bytes--i.e., Gigabytes--in size. Much of this results from users generating and accumulating large file sets which have aged, but which are considered valuable for future use.
Conventional options for increasing data storage capacity are often excessively costly or inconvenient to implement. Additional file servers can be incorporated into an existing computer network, but each additional server has a high incremental cost and adds to the administrative burden of managing the network.
Alternatively, the additional data can be stored offline on magnetic tape or other archival storage media. The economic costs associated with online storage of large data accumulations often compel system administrators to periodically remove apparently inactive data from online file server environment, archiving it to magnetic tape or an equivalent medium. This offline storage is much more economical than magnetic disk storage, and the use of such storage has grown commensurately with file server capacity. In 1987, a typical file server contained 105 Megabytes of data for each workstation it supported. In 1988, this typical storage level grew to 155 Megabytes, a 47% increase in one year. The typical number of archival data tapes grew from 47 to 65 per server in that same period.
While offline storage is economical, management of such offline tape libraries is often labor-intensive and error prone. Moreover, conventional archiving techniques share a number of deficiencies. For example, the selection of information to be archived is often arbitrary. Secondly, in many computing systems, such as those running under the UNIX operating system, no ready means is provided for cataloging information stored offline, requiring users to manually track their own information files. Related to these archival problems are those of migrating data--transferring computer data files, typically on off-line media, to an on-line medium, so that the files can be read and utilized by computers or processors in the network. Workstation users experience significant delays when it becomes necessary to retrieve information from the offline volumes. These delays are both frustrating and expensive.
In view of these and other problems presented by prior art data storage, the object of this invention is to provide digital data storage systems having the speed and convenience of online data storage, and the economy and large capacity of offline storage.
It is a further object of the invention to provide digital data storage systems that incorporate improved data migration methods.
Another object of the invention is to provide a data migration system that reduces the time associated with conventional data migration schemes, to the point that data migration occurs in a manner that is "transparent"--i.e., not discernable to the user.